{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "51466c8d-8ce4-4b3d-be4e-18fdbeda5f53",
   "metadata": {},
   "source": [
    "# How to stream messages from your graph\n",
    "\n",
    "LangGraph Cloud supports multiple streaming modes. The main ones are:\n",
    "\n",
    "- `values`: This streaming mode streams back values of the graph. This is the **full state of the graph** after each node is called.\n",
    "- `updates`: This streaming mode streams back updates to the graph. This is the **update to the state of the graph** after each node is called.\n",
    "- `messages`: This streaming mode streams back messages - both complete messages (at the end of a node) as well as **tokens** for any messages generated inside a node. This mode is primarily meant for powering chat applications.\n",
    "\n",
    "\n",
    "This guide covers `stream_mode=\"messages\"`.\n",
    "\n",
    "In order to use this mode, the state of the graph you are interacting with MUST have a messages key that is a list of messages.\n",
    "Eg, the state should look something like:\n",
    "\n",
    "```python\n",
    "from typing import TypedDict, Annotated\n",
    "from langgraph.graph import add_messages\n",
    "from langchain_core.messages import AnyMessage\n",
    "\n",
    "class State(TypedDict):\n",
    "    messages: Annotated[list[AnyMessage], add_messages]\n",
    "```\n",
    "\n",
    "OR it should be an instance or subclass of `from langgraph.graph import MessageState` (`MessageState` is just a helper type hint equivalent to the above).\n",
    "\n",
    "With `stream_mode=\"messages\"` two things will be streamed back:\n",
    "\n",
    "- It outputs messages produced by any chat model called inside (unless tagged in a special way)\n",
    "- It outputs messages returned from nodes (to allow for nodes to return `ToolMessages` and the like"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "521d975b-e94b-4c37-bfa1-82d969e2a4dc",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langgraph_sdk import get_client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "27a1392b-86c3-464e-99a8-90ffc965f3ec",
   "metadata": {},
   "outputs": [],
   "source": [
    "client = get_client()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "714e9f92-86b4-4cd8-9d68-cfc45d56ed2c",
   "metadata": {},
   "outputs": [],
   "source": [
    "assistant_id = \"agent\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "56aa5159-5583-4134-9210-709b969bda6f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'thread_id': 'e1431c95-e241-4d1d-a252-27eceb1e5c86',\n",
       " 'created_at': '2024-06-21T15:48:59.808924+00:00',\n",
       " 'updated_at': '2024-06-21T15:48:59.808924+00:00',\n",
       " 'metadata': {}}"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "thread = await client.threads.create()\n",
    "thread"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "147c3f98-f889-4f05-a090-6b31f2a0b291",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "runs = await client.runs.list(thread[\"thread_id\"])\n",
    "runs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "040795c6-5d9f-4729-9132-f3b0f94d9e94",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Helper function for formatting messages\n",
    "\n",
    "\n",
    "def format_tool_calls(tool_calls):\n",
    "    if tool_calls:\n",
    "        formatted_calls = []\n",
    "        for call in tool_calls:\n",
    "            formatted_calls.append(\n",
    "                f\"Tool Call ID: {call['id']}, Function: {call['name']}, Arguments: {call['args']}\"\n",
    "            )\n",
    "        return \"\\n\".join(formatted_calls)\n",
    "    return \"No tool calls\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7da70e20-1a4e-4df2-b996-1927f474c835",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Metadata: Run ID - 1ef2fe5c-6a1d-6575-bc09-d7832711c17e\n",
      "--------------------------------------------------\n",
      "Invalid Tool Calls:\n",
      "Tool Call ID: call_cg14F20jMBqWYrNgEkdWHwB3, Function: tavily_search_results_json, Arguments: \n",
      "--------------------------------------------------\n",
      "Tool Calls:\n",
      "Tool Call ID: call_cg14F20jMBqWYrNgEkdWHwB3, Function: tavily_search_results_json, Arguments: {}\n",
      "--------------------------------------------------\n",
      "Tool Calls:\n",
      "Tool Call ID: call_cg14F20jMBqWYrNgEkdWHwB3, Function: tavily_search_results_json, Arguments: {}\n",
      "--------------------------------------------------\n",
      "Tool Calls:\n",
      "Tool Call ID: call_cg14F20jMBqWYrNgEkdWHwB3, Function: tavily_search_results_json, Arguments: {'query': ''}\n",
      "--------------------------------------------------\n",
      "Tool Calls:\n",
      "Tool Call ID: call_cg14F20jMBqWYrNgEkdWHwB3, Function: tavily_search_results_json, Arguments: {'query': 'current'}\n",
      "--------------------------------------------------\n",
      "Tool Calls:\n",
      "Tool Call ID: call_cg14F20jMBqWYrNgEkdWHwB3, Function: tavily_search_results_json, Arguments: {'query': 'current weather'}\n",
      "--------------------------------------------------\n",
      "Tool Calls:\n",
      "Tool Call ID: call_cg14F20jMBqWYrNgEkdWHwB3, Function: tavily_search_results_json, Arguments: {'query': 'current weather in'}\n",
      "--------------------------------------------------\n",
      "Tool Calls:\n",
      "Tool Call ID: call_cg14F20jMBqWYrNgEkdWHwB3, Function: tavily_search_results_json, Arguments: {'query': 'current weather in San'}\n",
      "--------------------------------------------------\n",
      "Tool Calls:\n",
      "Tool Call ID: call_cg14F20jMBqWYrNgEkdWHwB3, Function: tavily_search_results_json, Arguments: {'query': 'current weather in San Francisco'}\n",
      "--------------------------------------------------\n",
      "Tool Calls:\n",
      "Tool Call ID: call_cg14F20jMBqWYrNgEkdWHwB3, Function: tavily_search_results_json, Arguments: {'query': 'current weather in San Francisco'}\n",
      "--------------------------------------------------\n",
      "Tool Calls:\n",
      "Tool Call ID: call_cg14F20jMBqWYrNgEkdWHwB3, Function: tavily_search_results_json, Arguments: {'query': 'current weather in San Francisco'}\n",
      "Response Metadata: Finish Reason - tool_calls\n",
      "--------------------------------------------------\n",
      "--------------------------------------------------\n",
      "AI: The\n",
      "--------------------------------------------------\n",
      "AI: The current\n",
      "--------------------------------------------------\n",
      "AI: The current weather\n",
      "--------------------------------------------------\n",
      "AI: The current weather in\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is over\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of \n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F).\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-s\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-south\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at \n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 k\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph).\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at \n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%,\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility is\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility is \n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility is 16\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility is 16 km\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility is 16 km (\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility is 16 km (9\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility is 16 km (9 miles\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility is 16 km (9 miles).\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility is 16 km (9 miles). The\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility is 16 km (9 miles). The UV\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility is 16 km (9 miles). The UV index\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility is 16 km (9 miles). The UV index is\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility is 16 km (9 miles). The UV index is \n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility is 16 km (9 miles). The UV index is 3\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility is 16 km (9 miles). The UV index is 3.\n",
      "--------------------------------------------------\n",
      "AI: The current weather in San Francisco is overcast with a temperature of 13.9°C (57.0°F). The wind is blowing from the south-southwest at 6.9 mph (11.2 kph). The humidity is at 81%, and the visibility is 16 km (9 miles). The UV index is 3.\n",
      "Response Metadata: Finish Reason - stop\n",
      "--------------------------------------------------\n"
     ]
    }
   ],
   "source": [
    "input = {\"messages\": [{\"role\": \"user\", \"content\": \"what's the weather in sf\"}]}\n",
    "config = {\"configurable\": {\"model_name\": \"openai\"}}\n",
    "\n",
    "async for event in client.runs.stream(\n",
    "    thread[\"thread_id\"],\n",
    "    assistant_id,\n",
    "    input=input,\n",
    "    config=config,\n",
    "    stream_mode=\"messages\",\n",
    "):\n",
    "    if event.event == \"metadata\":\n",
    "        print(f\"Metadata: Run ID - {event.data['run_id']}\")\n",
    "        print(\"-\" * 50)\n",
    "    elif event.event == \"messages/partial\":\n",
    "        for data_item in event.data:\n",
    "            if \"role\" in data_item and data_item[\"role\"] == \"user\":\n",
    "                print(f\"Human: {data_item['content']}\")\n",
    "            else:\n",
    "                tool_calls = data_item.get(\"tool_calls\", [])\n",
    "                invalid_tool_calls = data_item.get(\"invalid_tool_calls\", [])\n",
    "                content = data_item.get(\"content\", \"\")\n",
    "                response_metadata = data_item.get(\"response_metadata\", {})\n",
    "\n",
    "                if content:\n",
    "                    print(f\"AI: {content}\")\n",
    "\n",
    "                if tool_calls:\n",
    "                    print(\"Tool Calls:\")\n",
    "                    print(format_tool_calls(tool_calls))\n",
    "\n",
    "                if invalid_tool_calls:\n",
    "                    print(\"Invalid Tool Calls:\")\n",
    "                    print(format_tool_calls(invalid_tool_calls))\n",
    "\n",
    "                if response_metadata:\n",
    "                    finish_reason = response_metadata.get(\"finish_reason\", \"N/A\")\n",
    "                    print(f\"Response Metadata: Finish Reason - {finish_reason}\")\n",
    "        print(\"-\" * 50)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "langgraph-example-dev",
   "language": "python",
   "name": "langgraph-example-dev"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
